For a variety of reasons, it is frequently desirable to record and evaluate the large number of information requests and responses handled by the Web server(s) at a given Web site. For this reason, conventional Web servers normally include a mechanism for compiling a log file which records information on every received HTTP request, including the domain name of the remote host making the request, an identification of the remote user, the date and time of the request, the request line exactly as received, the status code returned to the client, and the length of the response returned. A number of publicly available log file analysis programs, as well as programs sold commercially such as netGenesis from netGenesis Corporation and WebTrends from WebTrends Corporation, provide mechanism for processing the log files produced by the server into a wide variety of management reports. In other cases, individual applications running on the Web server are programmed to record transaction information in a proprietary format which, while serving the needs of that application, is of little use to the Web site administrator concerned with the performance of the Web site as a whole.
There is a need, however, for a more robust mechanism for capturing data which describes selected desired content of individual request/response exchanges between Web browsers and Web servers independently of the data saved by individual applications or Web servers.